Codecov Report

Merging #118 into master will decrease coverage by 0.6%.
The diff coverage is 73.1%.

@@            Coverage Diff             @@
##           master     #118      +/-   ##
==========================================
- Coverage   85.53%   84.92%   -0.61%     
==========================================
  Files          62       62              
  Lines        4113     4200      +87     
  Branches      401      412      +11     
==========================================
+ Hits         3518     3567      +49     
- Misses        194      221      +27     
- Partials      401      412      +11

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9b6cf23...cb79ddb. Read the comment docs.

Thank you for fixing the memory problem!

Makes cljam.io.sequence/read-all-sequences lazy per chromosome. This laziness level doesn't have bad effect on performance.

I tested with criterium/quick-bench with full hg38.fa and got following results.

benchmarking code

(require '[criterium.core :as c])
(require '[cljam.io.sequence :as cseq])
(c/quick-bench (with-open [r (cseq/reader "/path/to/hg38.fa")]
  (dorun (cseq/read-all-sequences r))))

results (master)

Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 13.639211 sec
    Execution time std-deviation : 114.565829 ms
   Execution time lower quantile : 13.570642 sec ( 2.5%)
   Execution time upper quantile : 13.832132 sec (97.5%)
                   Overhead used : 1.164590 ns

results (fix/low-memory-sequence-io)

Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 21.326905 sec
    Execution time std-deviation : 38.759879 ms
   Execution time lower quantile : 21.283963 sec ( 2.5%)
   Execution time upper quantile : 21.366944 sec (97.5%)
                   Overhead used : 1.165616 ns

So there's about 50% (about 7-8 secs in practical) performance degradation.
Since sequential read was originally dedicated to low-level performance-intensive tasks, I think it's on the edge of tolerance levels.

I'm sorry for bothering you but please let me hear your opinion about this.
Thank you.

Thank you for pointing. The cause is boxing. I've fixed it by adding a primitive hint.

Performance of the new/old implementations seems to be roughly the same in my environment. Please check it again.

Thank you! 👍

The benchmark seems OK.

Evaluation count : 6 in 6 samples of 1 calls.
             Execution time mean : 13.247953 sec
    Execution time std-deviation : 113.367013 ms
   Execution time lower quantile : 13.182024 sec ( 2.5%)
   Execution time upper quantile : 13.436171 sec (97.5%)
                   Overhead used : 1.336517 ns

Thanks!